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(54) PREDICTION CODING SYSTEM FOR MOVING PICTURE 

(57)Abstract: 

PURPOSE: To suppress generation of information at a background in a 
frame even if panning or vibration of a camera takes place and to 
improve the coding efficiency by compensating the entire movement of 
the picture frame so as to apply background extraction or background 
prediction. 

CONSTITUTION: A moving vector detection circuit 8 detects an inter- 
frame moving vector 2 for each block comprising plural picture elements 
or each picture element for each frame from an input picture signal 1 
and uses the maximum likelihood as a global vector 3. An inter-frame 
prediction circuit 9 uses the moving vector 2 supplied from a moving 
circuit 8 to output a movement compensation inter-frame prediction 
value 4. Moreover, a background prediction background revision circuit 
10 uses the vector 3 supplied from the circuit 8 to output a background 
prediction value 5, and the value of the picture element of the 
background picture as a background prediction of a 1st frame picture 
element. When the global vector has a fraction component, the circuit 10 
calculates and outputs the background prediction value 5 of the picture 
element from the picture element in the vicinity of the position on the 
background picture with interpolation and prediction. A changeover 

switch 1 1 selects any of plural predictions, takes a difference 6 from the input signal 1 and the result is coded 
by a code conversion circuit 12. 
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(Effects) 

The prediction coding system of a moving picture that uses background prediction 
according to the present invention is described as follows: First, an inter-frame motion vector 
of an input image is detected on a per pixel basis or a per block of pixels basis. Then, a 
detected inter-frame motion vector is used to obtain a global vector that represents the motion 
of an entire image frame. The maximum likelihood value of the motion vector may be 
defined as the global vector, for example. Next, from among prediction values obtained by 
the both prediction modes of inter-frame prediction and background prediction, selected is a 
prediction value having the least absolute value of difference from an input image signal. 
Then, the prediction mode signal and the prediction difference value of a selected prediction 
value are encoded. Alternatively, in the prediction mode selection, an optimal prediction 
mode is estimated for a pixel to be encoded, with reference to a prediction mode whereby the 
absolute value of the predicted difference of an adjacent encoded pixel becomes the smallest. 
With a global vector GV=(GVx, GVy), the background prediction value of a pixel P(X,Y) is 
the value of a pixel Pbg(X-GVx,Y-GVy) of the background image. This global vector is now 
used to update a background image on a per frame basis As shown in Fig. 4, when the 
global vector of the i-th frame with respect to the (i-l)-th frame is GV=(GVx, GVy), then a 
position represented by a pixel Pi (X,Y) of the i-th frame corresponds to Pi-l(X-GVxY-GVy). 
Therefore, when a difference between the value of the pixel Pi(X,Y) and the value of the pixel 
Pi-l(X-GVx,Y-GVy) is smaller than a preset threshold value, then the value of the pixel 
Pbg(X,Y) of the background image is updated by the precious value of Pbg(X-GVx,Y-G Vy) 
and the previous value of Pi(X,Y) before updating, considering that Pi(X,Y) represents a 
background. In this situation, if the value before updating of Pbg(X-GVx, Y-GVy) has not yet 
been set, then the value of Pi (X, Y) is assigned to the value of Pbg(X, Y). When this update is 
not performed on a per frame basis, however, there is a need of using the value of a pixel 
Pbg(X-2GVx,Y-EGVy) of a background image as the background prediction value of the 
pixel Pi(X,Y). Note that SGV=(EGVx,2GVy) is a cumulative total value of global 
vectors from previous background image update to the current frame. 

A prediction coding system of an image sequence of consecutive frames using 
background prediction according to the present invention may be described as follows. First, 
prior to a series of operations of encoding, a large background image, which covers the entirety 
of background portions represented by the respective pixels in an image sequence, is set on 
reference coordinates. Likewise, a frame position vector that represents the motion position 
of an image frame on the reference coordinates is assigned in advance on a per image basis of 
an image sequence. Then, background prediction is performed by using the background 
image and the frame position vector. As shown in Fig. 5, when the frame position vector in 
the i-th frame is FVi=(FVxi,FVyi), then the position of a pixel Pi(X-FVxi,Y-FVyi) in the i-th 
frame corresponds to (X,Y) on the reference coordinates where the background image is set. 
Therefore, the background prediction value of the pixel Pi(X-FVxi,Y-FVyi) is the value of the 
pixel Pbg(X,Y) of the background image. 

A method of extracting background from an image sequence of consecutive frames 
according to the present invention may be described as follows. First, the frame position 
vector representing the motion position of an image frame on the reference coordinates is 
detected on a per image basis. Then, a background image is generated on the reference 
coordinates by using the frame position vector. It is assumed here that the frame position 
vector in the i-th frame is FVi=CFVxi ) FVyi), as shown in Fig. 5. Now, when 
Pi(X-FVxi,Y-FVyi) in the i-th frame is determined to represent a background, the value of 
Pi(X-FVxi,Y-FVyi) is extracted as the background data of the position (X,Y) on the reference 
coordinates. Fig. 5 shows a concrete example: When the frame position vector in the i-th 
frame is FV(i-l)=(FVx(i-l),FVy(i-l)), the position of a pixel Pi-l(FVx(i-l),FVy(i-l)) in the 
(i-l)-th frame and the position of the Pi(X-FVxi,Y-FVyi) in the i-th frame are commonly (X,Y) 
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on the reference coordinates. Given this fact, when a difference between the value of 
Pi-l(X-FVx(i-l),Y-FVy(i-l)) and the value of Pi(X-FVxi,Y-FVyi) ) is smaller than a 
predetermined threshold value, then it is determined that a background is represented. Then, 
the value of Pi(X-FVxi,Y-FVyi) is extracted as the background data of the position (X,Y) on 
the reference coordinates. Next, a background image is generated based on a plurality of 
background data extracted from the entire image sequence. If the values of the respective 
elements of the background data vary widely, then elements having large differences from the 
mean value of the above-mentioned data are screened out as inappropriate data. Then, 
background data having elements that vary little is used to obtain its absolute value. Then, 
the absolute value is assigned to the value of the pixel Pbg(X,Y), thereby generating a 
background image. 

The aforementioned effects of the present invention allow for the application of 
background extraction and background prediction in the midst of panning. It is also possible 
to reduce the amount of information generated hi the background portion of an image. Hence, 
coding efficiency may be improved. 
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